Transient Growth of Accelerated Optimization Algorithms

Optimization algorithms are increasingly being used in applications with limited time budgets. In many real-time and embedded scenarios, only a few iterations can be performed traditional convergence metrics cannot to evaluate performance these nonasymptotic regimes. this article, we examine the transient behavior of accelerated first-order optimization algorithms. For convex quadratic problems, employ tools from linear systems theory show that growth arises presence nonnormal dynamics. We identify existence modes yield an algebraic early quantify excursion optimal solution caused by modes. strongly smooth utilize integral constraints establish upper bound on magnitude response Nesterov’s algorithm. both Euclidean distance between variable global minimizer rise peak proportional square root condition number problem. Finally, for problems large numbers, demonstrate tightness bounds derive up constant factors.